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1 Introduction 



The origin of fermion masses and mixings in the Standard Model is still as 
mysterious as when the first elementary particles were being discovered. Un- 
like the couplings of fermions to spin one particles which are now understood 
in terms of Yang- Mills interactions, Yukawa couplings still await such a level 
of understanding. The lack of a first principle explanation for their patterns 
have led theorists to devise elaborate schemes [H 12 01 El, none of which 
(including our owns [HIIZI) are particularly convincing. A less ambitious ap- 
proach, couched in the language of low energy effective field theories, is that 
advocated by Froggatt and Nielsen jH] (FN): the hierarchies of masses and 
mixings stem from higher dimensional operators which, when evaluated in 
the desired vacuum, yield effective Yukawa interactions of the right strengths. 
This approach organizes the dimensions of these operators in terms of hith- 
erto unknown U{1) charges. The result is that the suppression level of a 
particular Yukawa coupling is related to its Froggatt-Nielsen charge. 

Among ideas and schemes that involve additional gauge symmetries (fla- 
vor or family symmetries) to explain this fermion hierarchy problem, models 
with f/(l) symmetries have been shown to be self-consistent, anomaly-free, 
and experimentally testable El 13 13 CDl • In these schemes, the addition 
of a particular symmetry that accommodates data is not derived from first 
principles; yet it could be very useful in hinting at its possible origin. 

Of the many versions of models of this type in the physics literature, those 
with chiral Froggatt-Nielsen charges are particularly restrictive, since their 
anomalies must be cancelled. We have investigated FN models with several 
charges: one is family independent and anomalous; its anomaly is cancelled 
by a dimension-five Green-Schwarz term at the cut-off together with 
anomaly-free family dependent FN charges which are responsible for the 
interfamily hierarchies. 

Specially daunting to these models has been the recent determination of 
the neutrino mass and mixing patterns. While the existence of neutrinos 
masses and mixings is perfectly natural and expected, the recent determi- 
nation of two large neutrino mixing angles poses further theoretical chal- 
lenges [?]. 

In this paper, we limit our investigation to the generation of anomaly-free 
family- dependent charges. We investigate the type of mechanisms capable of 
generating such symmetries. 

Cancelling a chiral anomaly is always done by adding new fermions. The 
easiest is to add fermions of the opposite chirality, the route Nature chooses 
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for QCD. The next to easiest is to add chiral fermions of different chiralities 
sucli tliat tlie result adds up to zero, Nature's choice for the hypercharge 
anomaly. Today we understand the latter as embedding the Standard Model 
fermions inside representations of anomaly-free groups. 

Central to anomaly cancellation are the groups. The anomaly free groups 
are well known: all Lie groups except SUn for n > 2. On the other hand, 
anomalies can occur only if the representations are complex, and so we fo- 
cus on anomaly-free groups with complex representations: the spinors of 
S0{2n + 6) and the complex representations of Eq. In fact the three fam- 
ilies of Standard Model fermions fit remarkably well in a spinor of SO{10) 
or a 27 of Eq. However the extra charges carried by these groups are not 
family- dependent, although they are anomaly-free over these representations. 

To find anomaly-free, family-dependent charges, we assume, in the spirit 
of Ref. [121, ^h^t gauge group at the Planck scale is replicated. Taking 
one copy of the same group G per family, for three families, the fundamental 
gauge group will be of the form 

Gpi = GxGxG (1) 

The group G should be simple, anomaly-free and should contain the group 
of the Standard Model, Gsm = SU{3) x SU{2) x f/(l)y. The fundamental 
group Gpi therefore contains Gsm x Gsm x Gsm- It has to be reduced to the 
diagonal Gsm, because Standard Model gauge interactions are flavor blind. 
Anomaly-free charges with opposite signs for different families can then be 
generated by order parameters which we call bi- or tri-chiral, to distinguish 
them from the usual bi-vector vacuum values. 

Symmetry breaking down to the Standard Model group proceeds in sev- 
eral steps. We assume that at some stage, there are U{1) family-dependent 
symmetries which dictate the orders of magnitudes of the Yukawa couplings. 
In this paper, we restrict the discussion to the generation of anomaly-free 
family-dependent phase symmetries. 

The question of family hierarchy reduces to a search for plausible unifying 
structures and the way to break them. The qualitative features of fermion 
masses and mixings are encoded in the underlying group structure and the 
breaking path. 

We will start with some examples that show how these ideas can be 
realized before working our way towards a realistic scheme. 
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2 Simple examples 

In order to fix our notation, and introduce key concepts, we start this inves- 
tigation with simple examples. 

SU{2) X SU{2) — > SU{2) 

We can first restrict the discussion to two families. The starting point is two 
copies of the simplest non-Abelian group, namely 

SU{2f X SU{2y , 

where a and f3 label the two copies. The two fermion families fall into the 
representations 

iPi ~ (2,1) , V2 ~ (1,2) . 

The order parameter that describes the symmetry breaking is taken to be a 
scalar field, transforming as a bi-fundamental representation 

H ~ (2,2) 

A suitable vacuum expectation value (vev) of H is able to trigger the breaking 
to the diagonal subgroup 

5C/(2)" X SU{2f — > SU{2Y+'^ 

because the order parameter contains a singlet of the unbroken subgroup 

2(^2 = 1 ©3 

After symmetry breaking, the fermions transform as doublets under the re- 
maining diagonal group. As long as the Higgs potential has no extra sym- 
metry, there is no left-over phase symmetry. The two families are not distin- 
guished by any family-dependent symmetry. 

SU{?>) X 5^(3) — > SU{2) X ^7(1) 

Our next example starts with the group 

SU{2>Y X SU{2,f , 
with two fermion families transforming as 

V'l ~ (3,1) = 3^*, ~ (1,3) = 3^. 
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We ignore for the moment the important question of anomaUcs. If the gauge 
group of the low-energy theory is the diagonal subgroup SU{2), different 
inequivalent symmetry breakings are possible. Consider first an order pa- 
rameter of the form Hi ~ (3,3); we call it a bi-vectorial order parameter, 
since the two copies appear the same way up to a conjugation. A suitable 
vev will trigger the breaking 

SUi3f X SUi3f — > SU{2f+^ X U{l)vc+v0 (2) 

where V""'^ is the Abelian factor in the embedding 

SU(3) D SU{2) X U{l)v 

The two fermion families are not distinguished after symmetry breaking by 
the remaining U{1) factor, since 

3" , 3^ — > 2i © 1_2 . 

In contrast, consider the bi-chiral order parameter, H2 ~ (3,3), which can 
produce the breaking 

5[/(3)" X SU{3f > SU{2Y+'^ X [/(l)ya_y/3 (3) 

In this case, the extra Abelian symmetry acts as a family symmetry with 
opposite charges for the two families 

3" 2i®l_2 
3^^ — > 2_i©l2 

We use this toy example to introduce a notation that enables us to catalog 
in a systematic way all the possible singlet directions that can be chosen by 
the scalar field during the symmetry breaking. A singlet under SU{2Y will 
be notated as l^i with its charge in the subscript. A direction singlet 
under the diagonal subgroup of SU (2)* x SU (2)-' but not under each group 
separately will be notated by V^^i^^j v^-v^)- ^ straightforward calculation of 

the product 3" 3'^, applied to the bi-vectorial order parameter yields the 
following SU{2)''+P singlet 

(3^,3'') ^ (2?©l°2)®(2^©lf) ^ © (4) 

The first singlet l(*o*2) leaves the family-independent C/(l)ya+y/3 unbroken. 
The second singlet, 01-2, leaves the larger subgroup SU{2)°' x SU{2Y x 
C/(l)ya+y/3 invariant. 
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The same analysis applied to the bi-chiral order parameter yields 

(3",3'^) ^ (2^er2)® (2^01^2) ^ ® (1-2®1^2) (5) 

It is clear that '^'^20) ^^"^ leaves the diagonal subgroup SU{2)°''^^ and the 
family- dependent phase symmetry U{l)ya_yfi unbroken. Although the fermions 
are chiral, this simple model is not realistic since the starting group is anoma- 
lous. 

GxG — > SU{?>) X ;7(1) 

It is more difficult to generate a diagonal 5^7(3), starting with two copies of 
a simple gauge group G. The maximal embeddings leading to a single SU (3) 
are 

SU{A) D SU{3) X f/(l) 4 = 3_i © I3 
Sp{6) D SU{3)xU{l) 6 = 3i©3_i 
SO{8) D SU{3) 8 = 8 

SU{6) D SU{3) 6 = 6 (6) 

G2 D SU{3) 7 = 3©3©1 

^6 ^ SU{3) 27 = 27 

Ey D 5f/(3) 56 = 28 28 

Only the ffist two embeddings contain an extra U{1) factor. 5*^(6) is vectorial 
under SU{3), and this chain cannot lead to a model with chiral fermions. 
This leaves 5'C/(4), putting anomahes aside. To avoid anomahes, one can 
embed SU{4:) into SO{7) 

S0{7) D SU{4) ; 7 = 6©1 , 

but then the 7 of SO (7) is vectorial, yielding vector-hke fermions under 

SU{3) 

SO{7) D SU{A) D SU{3)xU{l); 7 = 32e3_2©lo 

Because of SU{3) triality, a bi-chiral order parameter does not contain an 
SU{3)°'^^ singlet. This suggests we consider three copies 

GxGxG — > SU{3) X U{1) 
We start with three copies 

,5c/(4)° X su{4f X sui^y , 
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together with three fermion famihes 

ifj, ~ (4,1,1), ^2 ~ (1,4,1), ^3 ~ (1,1,4). 
In this case, the tri-chiral order parameter 

H ~ (4,4,4) 

is capable of breaking to the diagonal SU{3). Using the decomposition, 

SU{A) D SU{3) X U{l)v ; 4 = 3i © 1-3 , 
and using SU{3) triality, 

3®3(g)3 D 1 

we see that the breaking 

SU{4r X SU{Af X SU{4y ^ 5f/(3)"+^+^ x f/(l)y_y, x U{l)v,-v, 

is obtained if the order parameter takes a vev along the singlet l^g^Q^J^^, where 
the three subscripts refer to the sum of the charges and their two differences. 
Hence this order parameter produces two family- dependent phase symme- 
tries. This yields the following fermions: three quarks, distinguished by their 
family charges, (1,0), (—1,1), and (0,-1), together with three singlets of 
charges (—3,0), (3,-3), and (0,3). This model is chiral, but riddled with 
anomalies. 



3 Towards a realistic scheme 

We learned from these simple but unrealistic examples that a good candidate 
for a replicated gauge group must have complex representations to describe 
chiral fermions, and must be anomaly-free. The candidate groups are then 
either orthogonal groups with complex spinor representations, or the excep- 
tional group Eq pni- Here, we limit ourselves to studying groups that appear 
in the sequence 

Eq d SO(IO) D SU{5) D SU{3) X SU{2) x f/(l)y (7) 
The complex representation 27 of -Eg decomposes itself as 



Eg D ^O(IO) X U{l)v' 


27 = 


16x9 


no_ 


2 © I4 


50(10) D SU{5) X U{l)v 


16 = 


lOiQ 




©I5 




10 = 








SU{5) D SU{3) X SU{2) X U{1)y 


10 = 


(3,2) 


1/3 © 


.(3,1)_4/3©(1,1)2 




5 = 


(1,2; 


)-lffi 


(3, 1)2/3 



(8) 
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SO{10) X SO{10) X SO{10) — > SU{5) x f/(l) 



We first consider tlie case 

50(10)" X 50(10)^ X 50(10)^ — y 5f/(5)"+^+^ x f/(l) . 
A tri-cliiral order parameter 

Hi ~ (16,16,16) 

can trigger the desired breaking to the diagonal SU{5), because the product 
16® 16® 16 contains a SU (5) singlet, namely 

16"®16'^®16T D 10?®5!3®5l3 D Si^ = I'^^^J^ (9) 

where the U{1) charges for the SU{5)'^~^^~^'^ singlet io^+z'+t are given in sub- 
script in the form [v'^+v^ +v'^ , v'^—v^, f f"^). The product 16(8)16(8)16 also 
contains a singlet under SU{5)^, as can be seen from the decomposition (jSJ. 
These singlets are listed in Table [TJ 

Table 1: Singlets of SU{5) in the product 16" (E) 16^ 16"^ and associated 
broken directions (a, b, c). Singlets obtained by permutation of the indices a, 
j3, 7 are omitted. 



SU{5) Singlet in 
16® 16(8) 16 


Unbroken 
non-Abelian Group 


Broken 
Direction 


I5 ® if ® I5 


Sf/(5)" X SU{5)f^ X SU{5)^ 


(1,1,1) 


(-5,4,4) 




(1,-3,-3) 



Because a singlet direction carries three charges, it will always leave two 
independent U{1) symmetries unbroken. If all the charges vanish, then a 
third independent f/(l) is also unbroken. If we associate a vector v = (a, b, c) 
in a three dimensional space to the linear combination aV^ + bV^ + cV^, 
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then for each singlet with at least one non-vanishing charge, the vectors v 
corresponding to the U(l) symmetries left unbroken by this singlet span a 
(hyper-)plane. We can therefore associate to such a singlet a vector perpen- 
dicular to this plane, and call it the broken direction. 

For example, the singlet Sia is invariant under any linear combination of 
C/(l)y/3_y7 and C/(l)3ya_,_y/3. Thercforc, its broken direction is given by a vec- 
tor perpendicular to (0, 1, —1) and (3, 1, 0), so we can take Via = (1, —3, —3). 

Among all the possible linear combinations of V", and V^, the sum 
yet _|_ yp _j_ yy j^^^ distinctive characteristic to be family-blind. It will 
be left unbroken by a singlet S if the vector (1, 1, 1) is perpendicular the 
broken direction vs associated to S. In other words, the U{1) symmetry 
corresponding to vs has to be traceless over the family index. For example, 
the singlet Sia does not leave the combination V"' + + V"' unbroken. 

We also notice that the initial gauge group is invariant under a permu- 
tation of the indices a, /5, 7. However, a singlet S can spontaneously break 
this permutation symmetry, corresponding to the group P3 = Zg, down to a 
smaller subgroup D. All the singlets obtained from S* by a permutation of 
the indices a, 7 belong to the same conjugacy class under the coset Zq/ D. 

For example, the singlet 5*1^ is invariant only under the exchange j3 ^ 
In other words, it breaks the symmetry Zq down to D — Z2. Therefore, 
we can construct two equivalent singlets S^p and Si^, obtained from Si^ by 
cyclic permutation of the indices a, /3, 7 

C 1 a+/3+7 Q a+/3+7 

>J10 — -'-(_5,^4,o) 5 — -'-(-5,0,-4) 

Their corresponding broken directions are Vij3 = (—3, 1, —3) and Vi^ = 
(—3, —3, 1), and are again obtained by cyclic permutation of the indices a, 

The appearance of a spontaneously broken discrete symmetry can be 
potentially harmful in the cosmoligical context, because it leads to the for- 
mation of domains and domain walls. However, if the scale at which the 
symmetry is broken is very high, which is the case here, their presence will 
be washed away during inflation. 

We can now combine several singlets in order to narrow down the unbro- 
ken symmetry. By this, we mean that the order parameter can yield a non 
zero vev in more than one SU{5) singlet. The U{1) factors left over are found 
as the intersection of the unbroken spaces for each of the non-zero singlet. 
For example, the vev < Hi >= Sia ® >S'i^ will trigger the breaking 

,50(10)'* X SO{10f X SO{wy — > ,SC/(5)"+^+^ X C/(l)y^ 
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with 

Yp = V^ + V^-\v^ (10) 
This yields three SU (5) famihes 

(5_3, lOi, I5); (5^3, 10,, 1 5); (62, 10_2/3, l-io/s)- 

The subscript refers to their Yp values. This family symmetry distinguishes 
the third family from the first two. 

Let us emphasize the question of anomalies. The Abelian charge Yp given 
by Eq. (fTTH) is our first example of an anomaly-free and family-dependent 
symmetry. The cancellation of anomalies is achieved by completing the 16, 
and is ensured because 5*0(10) is an anomaly- free group. Therefore, the 
inclusion of all three right-handed neutrinos is necessary. If we take the SU{5) 
representations separately, the mixed anomaly coefficients (Yp SU{5) SU{5)) 
will not vanish, but the contributions from 10, 5 and 1 compensate each 
other. 

The mixed anomalies will only vanish over each SU{5) representation in 
the case where Yp is traceless. This can be achieved by taking a tri-vectorial 
order parameter instead of a tri-chiral one 

H2 ~ (45,45,45) 

With the branching rule 

^O(IO) D SU{5) xU{l)v ; 45 = 24o e 10_4 TO4 + lo (11) 

we can obtain singlets of the type 1" (g) 1^ ® 1"^, 1"+^ ® I'*' and I'^+f^+'y, They 
are listed in Table |2l together with their broken direction. 

Notice that the adjoint representation has the peculiarity that its tensor 
product with itself always contains a singlet and an adjoint representation. 
Therefore, in the case of 5*0(10), we have 

45"^® 45^ D 1; 45" 45^^ ® 45^ D 1 (12) 

These singlets have a zero charge under the diagonal U{l)v, t/(l)v"+v'' ^ind 
U{l)Ya^Yi3_^_yj respectively, but do not have a charge under the orthogonal 
combinations (and hence, these are broken). They are of course also singlets 
under the respective diagonal SU{5). However, using the branching rule (jllll . 
we can actually construct singlets contained in the tensor product of adjoint 
representations of 5f/(5), which have well-defined (and zero) charges under 
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Table 2: Singlets of SU{5) in the product 45" 45^ ® 45''' and associated 
broken directions. 



SU{5) Singlet in 
45 (g) 45 (g) 45 


Unbroken 
non-Abelian Group 


Broken 
Direction 


In ® In ® In 


SU{5)'^ X SU{5)l^ X 5C/(5)^ 


None 


-■-(0,0) ^ -"-o 


,SC/(5)"+^ X SU{5y 


None 


1 (5?) 1 ^ 
-■-(0,8) -"-O 


SU{bY+^ X ,SC/(5)T 


(1,-1,0) 


-^(0,0,0) 




None 


a+/3+7 
-^(0,8,0) 




(1,-1,0) 



the orthogonal linear combinations of y s. As mentioned previously, singlets 
for which all the charges vanish have no associated broken direction. 

The singlet 5*2 = l^'o^s'^J)^ has the desired features in order to give rise to 
a traceless family symmetry. The U{1) corresponding to its broken direction 
is traceless, therefore, it triggers the breaking 

50(10)" X 50(10)^ X 50(10)^ — > SUibY^^^^ X U{l)x X ^7(l)y^ 

with 

The charge X is family-blind while the charge Yp is traceless. This yields 
the following three chiral families along with their Yp charges 

(5_3, lOi, I5); (5_3, lOi, I5); (5 6, 10_2, l_io). 

As emphasized earlier, we notice that the mixed anomaly coefficients 
(Yp SU{b) SU{b)) between SU{5) and the traceless family charge Yp vanish 
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over each SU(5) representation. The charge Yp is said to be non-anomalous 
in the language of Froggatt and Nielsen. In contrast, the anomaly coefficients 
{X SU{5) SU{5)) for a given SU{5) are necessarily different from zero, but 
are family- independent. The charge X is said to be anomalous. 

The anomaly coefficient {Yp X X) also vanishes because Yp is traceless. 
Finally, the anomalies (YfYfYf), {XYpYp) and {XXX) can differ from 
zero for a given SU{5) representation, but they don't involve the Standard 
Model group or charges. As before, a complete cancellation of the anomalies 
is achieved by completing the 16 of 5*0(10). 

In the case that we have considered so far, we were able to construct a fam- 
ily symmetry that is anomaly-free, and get three families of chiral fermions 
with different family charges. However, the breaking of 5*0(10) down to 
SU{5) lowers the rank by only one unit, which does not leave enough room 
to build a family symmetry that can accommodate the observed phenomenol- 
ogy of fermion masses. This can be achieved by upgrading to Eq, as we are 
ready to see now. 



EexE^xEe — > SU{5) x ^(1) 

A further complication arises here, because SU{5) x U{1) is not a maximal 
subalgebra of E^. Two chains of maximal subalgebras can lead from Eq to 
SU{5), namely (leaving aside the possible U{1) factors) 

Eg d 50(10) D SU{5) (13) 

which is the chain we already considered, or 

Eq d 5C/(6) X SU{2) D SU(5) x SU(2) (14) 

In what follows, we will consider order parameters which are tri-chiral, tri- 
vectorial, bi-chiral and bi-vectorial 

Hi ~ (27,27,27) 
H2 - (78,78,78) 

H-i ~ (27,27, 1) (15) 
H4 ~ (27,27, 1) 

The irrep 27 contains a singlet of 50(10), but no singlet under SU{6) x 
SU{2), 

Ee D SU{6) X SU{2) ; 27 = (6, 2) ® (15, 1) 
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nor under SU{5) x SU{2) as we have 

SU{6) D SU{5)xU{l); 6 = 5i © 1_5 ; 15 = 5^4 102 

However, we can see that it contains two distinct singlets of SU{5). The way 
to understand this is the following. By making suitable linear combinations, 
one is a singlet under S'O(IO), and the other one is singlet under SU{5) x 
U{l)v but not under 50(10). We will designate these singlets by l(o,4) and 
1(5^1) respectively. They are distinguished by their charges (f , v') under U{l)v 
and f/(l)y'. 

Similarly, the irrep 78 decomposes itself as 

D SO{10) X U{l)v' ; 78 = 45o © 16_3 © T63 © lo 

Eg D SU{6)xSU{2)- 78 = (1, 3) © (35, 1) © (20, 2) 

Therefore, it contains one singlet of 50(10) but no singlet under SU{6) x 
SU{2). Using the branching rules 

SU{Q) D SU{5)xU{l); 20 = lO^affiTOa 

35 = lo © 56 © 5_6 © 24o , 

we further see that 78 does contain a singlet under SU{5) x SU{2). We 
can also see that it contains four singlets under SU{5) alone, which is in 
agreement with what is obtained using the chain of maximal subalgebras ()13|). 
because 

50(10) D SU{5)xU{l)v; 16 = lOi© 5^3 15 

45 = 24o © 10_4 © TO4 + lo 

Two of these singlets have a distinct non-zero charge V, they are designated 
by 1(5,3) and 1(5,-3), following our notation. The two remaining singlets have 
zero V and V charges. By making suitable linear combinations, one is singlet 
under 50(10), and the other one is singlet under 5f/(5) x 5f/(2). We will 
designate them collectively by 1(0,0), which is sufficient for the discussion of 
U{1) factors. The implications of an extra SU{2) factor in the framework of 
Froggatt and Nielsen is beyond the scope of the present paper, and will be 
described elsewhere. 

Following our method, a singlet under the diagonal 5f/(5) will be associ- 
ated with a broken direction in a six-dimensional space, v = (a, b, c, a', b', c') 
corresponding to the combination aV"' + bV^ + cV^ + aV" + b'V^ + dV^ . 
Many more possibilities of 5[/(5) singlet directions arise in this case. Lists 
of inequivalent singlets (under permutation of the indices a, (3, 7) and their 
associated broken directions are given in the Tables El to 
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4 Physical Implications 



4.1 More on anomalies 

As noticed earlier, singlets that leave the family-blind combinations unbro- 
ken are of particular interest. They trigger a symmetry breaking in which 
the U{1) symmetries are factorized into a family-blind part X and a family 
dependent part Yp which is traceless over the family index. This decompo- 
sition, in turn, enables a "multi-layered" anomaly cancellation, which is a 
central ingredient in the construction of Froggatt-Nielsen-type models. 

The first layer is given by the cancellation of all the possible anoma- 
lies involving only Standard Model groups over the fermion content of the 
Standard Model. This "accidental" cancellation is now understood in terms 
of embeddings, because the fermions of the Standard Model have the right 
quantum numbers to fit into the representations 10 and 5 of SU{5), which 
in turn, complete the 16 of 5*0(10) when a right-handed neutrino, singlet 
under SU{5), is added. 

In the second layer, there will be mixed anomalies between X or Yp 
and Standard Model groups Gsm- Because the family charge Yp com- 
mutes with SU{5) in our framework, it is sufficient to consider the mixed 
anomalies between X or Yp and SU{5). Anomaly coefficients with a sin- 
gle SU{5), namely {X X SU{5)), {XYpSU{5)) and {YpYpSU{5)), vanish 
over any representation of SU{5). The anomaly coefficient {Yp SU{5) SU{5)) 
vanishes over the family index because Yp is traceless. The anomaly coeffi- 
cients {X SU{5) SU{5)) differ from zero - X is anomalous - but are family- 
independent. In the low-energy point of view of effective theories of the 
Froggatt-Nielsen type, they are cancelled through a dimension five term 
at the cut-off, in the so-called Green-Schwarz anomaly cancellation mech- 
anism |llj . 

In the third layer, the remaining anomaly coefficients do not involve Gsm 
(or SU{5)). The anomalies (X Yp Yp) and {XXX) can also be compensated 
by the Green-Schwarz mechanism. The coefficient {Yp X X) vanish because 
Yp is traceless, so that the only non-vanishing coefficient that needs to be 
compensated is {YpYpYp). 

This last cancellation can be achieved by adding new matter fields which 
don't carry any Standard Model charge. Their presence in the effective the- 
ory is solely for the sake of anomaly cancellation, and does not modify the 
observable phenomenology. 

Of course, in the top-down approach used in our framework, we know how 
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the question of anomaly cancellation is resolved. The fermion content of the 
Standard Model is upgraded to the 27 of Eq by the addition of vector-like 
matter under SU{5), 5 ® 5, and two singlets. Therefore, it is not surprising 
that the theory looks anomalous when only the Standard Model fermions are 
taken into account! 



4.2 Predictions for fermion masses and mixings 

Among the possible linear combinations of V and V, V"' + + and 
y>a _|_ y/13 _|_ yf^y family-blind. Therefore, at least three singlets are needed 
in order to fix completely the traceless family symmetry Yp. If this is the 
case, the interfamily mass ratios are completely determined by the family 
charges Yp of the fermions, independently of the dynamics which further 
breaks the family symmetry. 

Let us briefly recall how the family symmetry is related to fermion mass 
hierarchies. To flx our notation, we will use the supersymmetric setup of 
Ref . jni Ej . The Yukawa couplings for quarks and charged leptons stem from 
invariants in the superpotential of the form QiUjHu, QidjHd, LiEjHd, where 
i and j are family indices. For neutrinos, after the see-saw mechanism has 
taken place, the effective Yukawa coupling is given by the quartic invariant 
LiLjHl In the presence of a family symmetry, these invariants can appear 
in the superpotential only if they are not charged under the extra family 
symmetries. If they are charged, they can still appear, but only as higher 
dimensional operators 




where J is a MSSM invariant, 6 is an order parameter, singlet under Gsm 
but charged under Yp, and M is the cut-off scale, usually taken as the string 
scale. The operator / has to be gauge invariant. The families symmetries 
are spontaneously broken when the 6 flelds get a vev, yielding the expansion 
parameter 



M 

As a result, the Yukawa coupling corresponding to the invariant lij will be 
suppressed by a power riij which is related to its family charge Yp 

Uij = -Yp{Iij) + est. (16) 

where the family-independent constant arises because of the anomalous charge 
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X. We can notice that these powers obey the sum rule 



In our framework, the family symmetry depends only upon V and V, and 
therefore commutes with SU{5). The structure of the mass matrices is then 
determined by the charges —1^(10) and —Yp{5), reordered and normalized 
to the heaviest family 

Q , u , e - y>(io) 

L , d — > - Yf(E) 

We have taken all the possible sets of three singlets that determine the 
family symmetry Yp, and derived the corresponding Yukawa structure. How- 
ever, thousands of different patterns can emerge. To reduce this number, 
we chose to enforce a phenomenological constraint. To make it strong and 
reliable, only the heavier fermions from the second and third families are 
involved. 

The measured fermion mass ratios scale as 

^ - ^ ~ A2 ; ~ A^ (17) 

mt rrib rrir 

where the expansion parameter A is of the order of the Cabibbo angle Ac — 
0.22. Therefore, we can use the constraint derived from the relations 

^ ~ ~ (18) 

After noticing that Eq. ()18|1 is compatible with a family symmetry commuting 
with S'[/(5), we see that it translates into 

Yf(^2) = 1f(53) (19) 

This relation, in turn, implies a large mixing angle for the atmospheric neu- 
trinos, as it is indeed observed 14] ! 

It turns out that the constraint (fT^ restricts severely the number of possi- 
ble patterns. Four 'scenarios' were considered, in which the order parameter 
that is spontaneously broken transforms as Hi, H2, H4 and as Hi or H^ in 
the last one (see Eq. (fTH|l ). We find that a bi-chiral or a bi- vectorial order 
parameter does not lead to a family symmetry with physical interest. The 
family charges for a tri-chiral or tri-vectorial order parameter that survive 
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from the constraint (|19p are listed in Tables 6 and 7. The interesting charges 
that give rise to a phenomenology compatible with all present data on quarks, 
charged leptons and neutrinos masses and mixings have been underlined. It 
can be noticed that all of them can be obtained with an order parameter 
transforming like H2- 

Most of these charges are only in weak agreement with the data. The 
mismatch lies in the predicted masses for the first generation. A light up 
quark mass is in conflict with the derivation of the correct CKM matrix. A 
further conflict comes from the neutrinos sector, where a mild Am? hierarchy, 
and a large solar mixing angle induce a heavier electron. 

In the following, we analyze in more details three of these possible charge 
assignments. 



Model A 

The charges —Yp{lQi) ~ (2,1,0) and — Yf(5) ~ (0,0,0) with expansion 
parameter A ~ give rise to the so-called anarchical model [15j, where 
hierarchical and mixing structure is totally absent from the neutrino mass 
matrix. The observed hierarchy between the Am^ for atmospheric neutrinos 
and the Am^ for solar neutrinos (for the LMA solution |17j ) 

^ ^ 10-' (20) 
Ami 

can still be obtained but resides in Nature's choice of the prefactor coeffi- 
cients. Large mixing angles for solar and atmospheric neutrinos are natural 
in this context, but the small value of the CHOOZ JH] mixing angle is prob- 
lematic. The model is otherwise in fair agreement with phenomenology. The 
predicted mass ratios are 



mt 


\8 ■ 

'^C ) 


me 
mt 


~ A^ 


rrid 
rrib 




rris 
rrib 


~ A^ 


me 
m^- 


/\e , 


m-r 


~ A^ 



However, we notice that the electron mass comes out too heavy compared to 
the r mass. A smaller ratio 

m^ 
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would be in better agreement with the measured masses. Moreover, some 
stretching in the prefactor coefficients is needed to reconcile the model with 
the structure of the CKM matrix pTS] . 

Model B 

The charges -Yf{10) ~ (3, 2, 0) and -Yf{5) ~ (2, 0, 0) with A ~ enable to 
reproduce the CKM matrix. The predictions for quarks and charged leptons 
are 





\6 . 

''c ) 


rric 


A^ 






nit 




mt 




rrid 


\4 . 

, 


nis 


A^ 






nib 




nib 




rrie 




rrif. 








nir 




nir 





with a heavier up quark. In the neutrino sector, the resulting Am? hierarchy 
is in agreement with Eq. ()2()|1 . the CHOOZ mixing angle is suppressed by a 
factor A^, but the solar mixing angle also turns out to be naturally small, 
which is less satisfying in view of the recent solar neutrino data ^Tj, and the 
KamLAND result [ISj. 

A phenomenologically better charge assignment would be — Yf(10) ~ 
(3,2,0) and — Yf(5) ~ (1,0,0) It can been incorporated in a consistent way 
in a FN model [7j, but could not be reproduced by symmetry breaking in 
the present approach. 

Model C 

Finally, the charges -^^(10) ~ (5,2,0) and -Yf(5) ~ (1,0,0) also with 
A ~ Ac can be in fairly good agreement with the observations. The predicted 
mass ratios are 



rriu 
mt 


A 


rric 

mt 


~ A^ 


rrid 
nib 




rris 
rrib 


~ \l 


rrie 
rrir 




mr 





We notice that the electron to r mass ratio have the correct order of mag- 
nitude, but the up and the down quarks appear a little light. However, it 
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has been pointed out that non perturbative QCD effects can also contribute 
to the masses of the hghtest quarks at low-energy, so that the actual masses 
and the mixings in the CKM matrix can be recovered (see ^HI for example). 

5 Conclusions 

The possibility that the fundamental gauge group at very high scales appears 
replicated in several copies, as inspired by the brane world, opens up new 
possibilities to understand the patterns in the low energy world. We have 
studied how the family mass hierarchy problem can be elucidated in this con- 
text through a group-theoretical approach. Imprints of symmetry breakings 
were recognized and systematically analyzed, although the dynamics which 
triggers the symmetry breaking is beyond the scope of the present work. 

The realistic scheme considered, based on Eq x Eq x Eq SU{5) shows 
interesting features. Although many different family symmetries can survive 
after symmetry breaking, a simple and reliable phenomenological constraint 
killed most of them. Moreover, only a few possibilities lead to mass pat- 
terns in accordance with observations. So it might be that our world has 
something exceptional rather than generic, that very particular dynamical 
conditions triggered such a symmetry breaking. Or it might also be that 
the true mechanism that Nature choose to order fermion masses is totally 
different. 

However, it is worth pointing that this approach does give rise to patterns 
in agreement with our world. Moreover, it appears that this path, for reasons 
explained throughout this paper, necessarily leads to the consideration of 
exceptional algebras. So Nature might indeed be exceptional... Anyhow, the 
idea that mass could be partially treated as a quantum number is a very 
attractive scheme that can help us organize and understand the legacy of the 
Standard Model. 

Acknowledgments 

This work is supported by the United States Department of Energy under 
grant DE-FG02-97ER41029. 



19 



References 

[1] L. Ibafiez, G. G. Ross, Phys. Lett. B 332, 100 (1994); Pierre Binetruy 
and Pierre Ramond, Phys. Lett. B 350, 49 (1995). 

[2] R. Barbieri, G. Dvali, L. J. Hall, Phys. Lett. B 377, 76 (1996); A. 
Aranda, C. D. Carone and P. Meade, Phys. Rev. D 65, 013011 (2002). 

[3] S. F. King and G. G. Ross, Phys. Lett. B 520, 243 (2001). 

[4] M. S. Berger and Kim Siyeon, Phys. Rev. D 64, 053006 (2001). 

[5] C. Froggatt and H.B. Nielsen, Nucl. Phys. B 147, 277 (1979). 

[6] P. Binetruy, N. Irges, S. Lavignac and P. Ramond, Phys. Lett. B 403, 38 

(1997) ; N. Irges, S. Lavignac and P. Ramond, Phys. Rev. D 58, 035003 

(1998) . 

[7] Fu-Sin Ling and Pierre Ramond, Phys. Lett. B 543, 29 (2002). 

[8] Ann E. Nelson and David Wright, Phys. Rev. D 56, 1598 (1997). 

[9] J. Sato and K. Tobe, Phys. Rev. D 63, 116010 (2001). 

[10] N. Maekawa, Prog. Theor. Phys. 107, 597 (2002). 

[11] M. Green and J. Schwarz, Phys. Lett. B 149, 117 (1984). 

[12] C. D. Froggatt, M. Gibson, H. B. Nielsen and D. J. Smith, IJMPA 13, 
5037 (1998). 

[13] R. Slansky, Phys. Rcpt. 79, 1 (1981); G. W. Anderson and T. Blazek, J. 
Math. Phys. 41, 4808 (2000); G. W. Anderson and T. Blazek, J. Math. 
Phys. 41, 8170 (2000). 

[14] The Super-Kamiokandc Collaboration, Phys. Rev. Lett. 85, 3999 (2000); 
T. Toshito for the SK Coll., Proceedings of the XXXVIth Rencontres 
de Moriond - Electroweak Interactions and Unified Theories, 36 (2001) 

[15] M.S. Berger and K. Siyeon, Phys. Rev. D 63, 057302 (2001). 

[16] T. Banks, Y. Nir, N. Seiberg, Yukawa Couphngs and The Origins of 
Mass (Conference Proceedings), 26 (1996). 



20 



[17] M. B. Smy for SK, Proceedings of the XXXVIth Rencontres de Moriond, 
[hep-ex/0106064i The SNO Collaboration, Phys. Rev. Lett. 87, 071301 
(2001); The SNO Collaboration, Phys. Rev. Lett. 89, 011301 (2002); 
The SNO Collaboration, Phys. Rev. Lett. 89, 011302 (2002). 

[18] The KaniLAND Collaboration, Phys. Rev. Lett. 90, 021802 (2003). 

[19] M. Apollonio et ai, Phys. Lett. B 338, 383 (1998); M. ApoUonio et ai, 
Phys. Lett. B 420, 397 (1998). 



21 



Table 3: Singlets of SU{5) in the product 27" 27^ ® 27^ and associated 
broken directions {a,b,c,a',b',c'). Singlets obtained by permutation of the 
indices ct, /3, 7 are omitted. 



SU{5) Singlet in 


Unbroken 
non-Abelian Group 


Broken 
Direction 


1(0,4) ® lfo,4) ® 1^0,4) 


50(10)" X SO{10f X 50(10)^ 


(0,0,0,1,1,1) 


Ifs,!) ® lfo,4) ® 1(0,4) 


SU{5)'^ X ,50(10)^ X 5O(10)T 


(5,0,0,1,4,4) 


Ifs,!) ® ifs,!) ® 1(0,4) 


,5C/(5)" X ,SC/(5)^ X SO{10)^ 


(5,5,0,1,1,4) 


1(5,1) ® lf5,l) 1(5,1) 


5t/(5)" X 5t/(5)^ X SU{5y 


(5,5,5,1,1,1) 


-^(0,-4,4,0) ^ -^(0,4) 


SU{bY+^ X ,5O(10)T 


(1,-1,0,-1,-1,2) 


-^(-5,-1,-1,3) ^ -^(0,4) 


SU{bY+P X ,SO(10)T 


(-3,-2,0,1,-2,4) 


-^(-5,-1,-1,3) ^ -^(5,1) 


5[/(5)"+^ X SU{b)^ 


(-3,-2,5,1,-2,1) 


1 (5?) 1 
-^(0,-4,4,0) ^ -L(5,l) 


SU{hY+^ X 5t/(5)^ 


(2,-2,5,-2,-2, 1) 


-^(-5,3,4,0,4,0) 
-^(0,0,-5,3,-4,0) 

-^(0,0,0,0,3,3) 

-, a+/3+7 
-^(5,-3,0,0,1,-3) 




(1,-3, -3,1,1,1) 
(-3,2,1,1,-2,1) 
(1,1,-2,1,1,-2) 
(2,2,1,-2,-2,1) 
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Table 4: Singlets of SU{5) in the product 78" 78^ ® 78^ and associated 
broken directions. Singlets obtained by permutation of the indices a, 7 
and conjugate singlets (with the sign of all charges reversed) are omitted, 
and G stands for either ,50(10) or SU{5) x SU{2). 



SU{5) Singlet in 
78 78 (0 78 


Unbroken 
non-Abelian Group 


Broken 
Direction 


1(5,-3) ® lf5,-3) ® 1(5,-3) 


5[/(5)" X 5[/(5)^ X SU{5)^ 


(5,5,5,-3,-3,-3) 


-■■(5,-3) <^ -"-(5,-3) -"-(-5,3) 


'^TK'^)'^ V '^TK'^)!^ V '?/"/"(' "it 




1(5,-3) ® lf5,-3) ® 1(0,0) 


5[/(5)" X 5C/(5)^ X 


(5,5,0,-3,-3,0) 


lf5,-3) ® lf-5,3) ® 1(0,0) 


SUib)" X 5^/(5)'^ X 


(5,-5,0,-3,3,0) 


lf5,-3) ® lfo,0) ® 1(0,0) 


SU{5)'^ xC^ xG^ 


(5,0,0,-3,0,0) 


-^(0,0,6,6) -^(0,0) 




(1,-1,0,1,-1,0) 


-^(0,0,-8,0) <^ -^(0,0) 
-•-(0,0,2,-6) ^ -"-(0,0) 


SU{5)"+^ X G^ 


(1,-1,0,0,0,0) 
(1,-1,0,-3,3,0) 


l(_5,3,-3,-3) <^ -^(0,0) 




(-4,-1,0,0,3,0) 


lfo,o) ® lfo,o) ® 1^0,0) 


G^ xG^ X G^ 


None 


-•-(0,0,0,0) -"-(0,0) 


SU{hY+P X 


None 


a+/3+7 
-•-(0,0,0,0,0,0) 




None 
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Table 5: Singlets of SU{5) in the product 78" O 78^^ 78^ and associated 
broken directions (continued). 



SU{5) Singlet in 
78 (g) 78 (g) 78 


Unbroken 
non-Abelian Group 


Broken 
Direction 


-^(0,0,6,6) <^ -^(5-3) 




(3, -3,5,3,-3,-3) 


-^(0,0-8,0) -^(5,-3) 




(-4,4,5,0,0,-3) 


-•-(0,0,2,-6) -"-(5,-3) 


SU{5)''+^ X 5t/(5)^ 


(1,-1,5,-3,3,-3) 


-■-(-5,3,-3,-3) -'■(5,-3) 




(-4,-1,5,0,3,-3) 


-^-(-5,3,-3,-3) ^ -^(-5,3) 




(-4,-1,-5,0,3,3) 


-^(10,6,0,0,-1,3) 




(3,3,4,3,3,0) 


-J a+/3+7 
-^(5,9,0,0,4,0) 




(3,3,-1,3,3,3) 


^ a+/3+7 
-^(5,-3,0,0,7,3) 




(4,4,-3,0,0,-3) 


-. Q+/3+7 
-^(-5,3,0,0,2,6) 




(-1,-1,-3,3,3,-3) 


1 Q+/3+7 
-^(0,0,5,-3,7,3) 




(4,-1,-3,0,3,-3) 
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Table 6 : Family charges (—1^^(10), — Yf(5)) normalized to the heaviest 
family (third family), obtained from Hi ~ (27, 27, 27). 



(1, 1, 0), (0, 0, 0) 
(1, 1, 0), (8, 0, 0) 
(3, 1, 0), (4, 0, 0) 



(1, 1, 0), (1, 0, 0) 
(2, 2, 0), (1, 0, 0) 
(3, 2, 0), (2, 0, 0) 



(1, 1, 0), (2, 0, 0) 
(2, 2, 0), (7, 0, 0) 
(4, 3, 0), (1, 0, 0) 



(1, 1, 0), (5, 0, 0) 
(3, 1, 0), (1, 0, 0) 
(5, 2, 0), (4, 0, 0) 



Table 7 : Family charges {—Yf{10),—Yf{5)) normahzed to the heaviest 
family (third family), obtained from H2 ~ (78, 78, 78). 



(1, 1, 0), (2, 0, 0) 

(2, 2, 0), (1, 0, 0) 

(3, 2, 0), (2, 0, 0) 

(4, 3, 0), (1, 0, 0) 

(5, 2, 0), (4, 0, 0) 

(7, 4, 0), (5, 0, 0) 



(1, 1, 0), (5, 0, 0) 
(3, 1, 0), (1, 0, 0) 
(3, 2, 0), (5, 0, 0) 
(4, 3, 0), (10, 0, 0) 
(5, 5, 0), (7, 0, 0) 
(8, 5, 0), (1, 0, 0) 



(2, 1, 0), (0, 0, 0) 
(3, 1, 0), (4, 0, 0) 
(4, 1, 0), (2, 0, 0) 
(5, 1, 0), (3, 0, 0) 
(6, 2, 0), (5, 0, 0) 
(9, 5, 0), (2, 0, 0) 



(2, 1, 0), (3, 0, 0) 
(3, 1, 0), (16, 0, 0) 
(4, 1, 0), (5, 0, 0) 
(5, 2, 0), (1, 0, 0) 
(6, 5, 0), (8, 0, 0) 
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